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Abstract 

We study the following sequential assignment problem on a finite graph G = {V,E). 

Each edge e € E starts with an integer value Ue > 0, and we write n = time t, 

1 < < < n, a uniformly random vertex u G E is generated, and one of the edges / incident 
with V must be selected. The value of / is then decreased by 1. There is a unit final reward 
if the configuration (0,..., 0) is reached. Our main result is that there is a phase transition: 
as n —>■ oo, the expected reward under the optimal policy approaches a constant cq > 0 
when {uejn : e £ E) converges to a point in the interior of a certain convex set TZg, and 
goes to 0 exponentially when {uejn : e £ E) is bounded away from TZg- We also obtain 
estimates in the near-critical region, that is when (rie/n : e £ E) lies close to &1Zg- We 
supply quantitative error bounds in our arguments. 

Keywords: phase transition, critical phenomenon, stochastic sequential assignment, Markov 
decision process, stochastic dynamic programming, discrete stochastic optimal control. 

1 Introduction 

Consider the following game (known in different versions [6], [111 Section 1.7]). Players start 
with a row of N empty boxes. In each of N rounds, a random digit is generated, and each player 
has to place it into one of the empty boxes they have. A player’s score is the N digit number 
obtained after the last round. The game is a special case of sequential stochastic assignment 
introduced by Derman, Lieberman and Ross [3]. In sequential assignment, there are N jobs 
with given values pi < ■ ■ ■ < pN that have to be assigned to N workers, as they appear in 
sequence. The z-th worker has ability Aj, where Ai,..., A^v are i.i.d. random variables from a 
given distribution F. The reward from assigning the job of value pi to a worker with ability x 
is piX, and the overall reward of the assignment is the sum of the individual rewards. The game 
mentioned at the start is recovered when pi = 10*“^, and Aj is uniform in {0,... , 9}. 

The paper [3] showed that there is a strategy that maximizes the expected score inde¬ 
pendently of what pi,...,pjv are. This strategy has the following form. There are numbers 
—oo = ao,n < ai,n < • • • < CLn-i,n < On.n = cCj n > 1, that Only depend on the distribution F, 
such that if there are n jobs remaining to be assigned, with values < • • • < and the next 
worker has ability x with ai-\^n ^ x < then the worker is assigned to the job with value p(. 
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Albright and Derman [T] showed, using law of large numbers type arguments, that when F 
is absolutely continuous, one has lim„^ooagn,n = 0 < g < 1, as n ^ oo. In particular, 

when the number n of jobs is large, a worker with ability x should be assigned to a job with rank 
approximately qn, where F~^{q) = x. Note that when F is discrete, this way of determining 
the asymptotics breaks down: when x is an atom of F, the graph of F~^ has a horizontal piece 
at height x. For large finite n, the value of q where the profile aqn,n crosses height x can be 
expected to be somewhere in the corresponding interval of constancy of F~^, and its precise 
location can be expected to be governed by large deviation effects. 

In order to motivate the subject of our paper, consider the following modification of the 
game mentioned at the beginning. Suppose that each digit can take the values 1,... ,k, with 
equal probability. Also suppose that the goal of the player is to maximize the probability of 
achieving the maximum possible score, that is to reach the unique final assignment consisting 
of k contiguous intervals of equal digits. Let r be the first time when all k numbers have 
occurred at least once. At time r, the empty boxes form k — 1 intervals of lengths ni,..., n^-i, 
where n — t = Yli=i interval has a box filled with i adjacent to it on the right, 
and a box filled with i + 1 adjacent to it on the left. It is plausible that there exist numbers 
0 = ai < a 2 < • • • < CKfc-i < = 1) such that for large n, under the optimal strategy, Uj/n ~ 

ccj+i — ai, i = 1,... ,k — 1. We will be interested in the following question. Suppose that an 
alternative position is imposed on the player, where the intervals have length n' (A+i -/3i)n', 
i = 1,..., /c — I, where 0 = /3i < /32 < • • • < h-i < /3fc = 1. What is the behaviour of the 
probability that the player can achieve the maximal score from this position? 

We show that the above probability displays a sharp transition in the limit n' —oo. When 
the vector (/3i+i — Pi : i = 1,... ,k — 1) lies in the interior of a certain convex set TZk, the 
probability approaches a positive constant, whereas it goes to 0 exponentially when the vector 
is at a positive distance from TZk. 

More generally, we consider the above transition on a general finite graph G = {fV, E) with 
vertices labelled l,...,k. The starting position is a vector (ue : e € E), and n = 

When a number 1 < i < k is rolled, one of the edges / incident with vertex i is selected by the 
player, and the value assigned to edge / is decreased by 1. We assign a final reward of 1 when 
the configuration (0,... ,0) is reached, and refer to this as ‘winning’. In the game described at 
the beginning, the graph is a path of length k — 1. 

We believe the study of this model is interesting for a number of reasons. 

1. Questions of reachability have been studied in control theory for a long time [lOl Sections 
19,20]. In our model, the controllable set TZg, that allows the player to reach the state 
(0,... , 0) with uniformly positive probability, has a simple characterization, which however 
involves the graph structure in a non-trivial way; see Eqn. m and Lemma 01 As we 
show, choosing the right control is only essential near dTZc. We believe our model, that 
is tractable on a general graph, is a useful example system to have in understanding the 
behaviour of discrete controlled systems with spatial structure near critical regions. Indeed, 
the main technical effort in this paper is getting estimates in the near critical region, that 
we do in Section [3l 

2. In deriving the optimal strategy for sequential assignment, Derman, Lieberman and Ross 
[3] used Hardy’s inequality, of which we have no analogue on graphs. Our proofs work 
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without knowledge of the optimal strategy, and only rely on martingale and Lyapunov 
function techniques, as well as an explicit relationship between TZg and available controls. 
Thus our arguments may be adaptable to other models. It may be that the transition 
phenomenon itself can be established with less effort, given more information on the opti¬ 
mal strategy (see for example Question [1] in Section 0]). Nevertheless, we believe that the 
quantitative bounds we derive are of independent interest. 

3. As the title of this paper suggest^ we view the transition studied in this paper as an 
instance of a critical phenomenon!^ While such transitions are ubiquitous in stochastic 
control, we found little in the literature that connects them with critical phenomena. We 
believe that such a point of view can be beneficial, and was indeed our original motivation 
for this study. Examples of works in the physics literature that address an interplay 
between controllability and network structure are PlZllla]. 

4. Further problems that are important for applications can be studied in our model or 
suitable modifications thereof. For example, we see no obvious distributed control, where 
vertices would only have local information about the graph structure. 

1.1 Definition of the model 

Throughout G = {V, E) will be a finite connected simple graph (without multiple edges or 
loops). We write k = \ V\, and assume \E\ > 2 (the case with one edge being trivial). We write 
degQ(u) for the degree of v G V, and degp{v) for the degree of v in the subgraph of G induced 
by the set of edges F C E. 

The state at time 0 < t < n is an integer vector N(t) = {Ne(t) : e G E), where the starting 
state is N(0) = n = (ng : e G E). Usually we will use capitalized letters for random variables or 
random processes, and lowercase letters for their possible values. We write n = 

Vi,.. .Vn G U be an i.i.d. sequence of vertices with P[Uj = u] = ■^, u G U, i = 1,..., n. If the 
player allocates V) to the edge e incident with V); the state is updated as 


N(t) = N(t — 1) — 1®, where 1® 



The gambler wins if N(n) = (0,... ,0) € N®, and looses otherwise. We denote by pg{t^) the 
probability of winning under the optimal strategy, when the starting state is n. This satisfies 



( 1 ) 


v&V 


known as the optimality equation [121 Section I.l], where e ~ u means that e is incident with v. 


We introduce some notation needed to state our main theorem. We write Sg for the prob¬ 
ability simplex in that is, the set of non-negative vectors x G such that = 1- 


reader unfamiliar with critical phenomena can find a good introduction in the short text [4]. We note that 
such familiarity is not required for understanding this paper. 
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We define 


d{F) = \{v eV : degp{v) = degG('y)}| , C F C E] 

TZg = i X € Sg : for all 0 C F C we have Xe > r > 

I e&F ) 

Xg = < X G Sg '■ there exists $ C F C E such that Xe < TdiF) > 

I I 


( 2 ) 


The letters ‘d’, ‘F’ and ‘X’ are intended to evoke ‘degree’, ‘reachable’ and ‘inaccessible’, as we 
explain. For any non-empty set F of edges, is the probability that the player receives a 
vertex that has full degree in F. Any such vertex must be allocated to one of the edges in F. For 
starting positions n = (rie : e € F) where the proportion of space available at the 

beginning is smaller than d{F)/k, the probability of winning goes to 0 (as n —oo). Therefore, 
from the region Ig the winning position is asymptotically inaccessible. On the other hand, as 
we show in Theorem [U if n = nx with x G Fc, then the winning position is asymptotically 
reachable from n. As we point out in Section [2711 the set TZg arises as the region of controllability 
for a simple (deterministic) linear control system associated to the game. It can be verihed that 
when G is a tree with k vertices {k > 3) TZg is a parallelepiped. As we will not need this fact, 
we omit the proof. 

Remark. The arguments we present in this paper are also applicable to the slightly more general 
model when Vi,... ,Vn are not uniformly distributed (but still i.i.d.). Suppose P[Fj = v] = 
with a probability vector p = (py : v G V) such that > 0 for all v G V. In this case TZg and 
Ig are replaced by 

T^G,p = s X G Sg : for all 0 C F C F we have Xe > Pv 

eSF i;:deg^('u)=degQ(i;) 

27g,p = < X G Sg ■ there exists tj) F F F E such that Xe < Pv 

e&F 'u:deg^(i;)=degc('u) 

As the required changes in the proofs are minor, but including them would burden the notation 
further, we state and prove the results only in the uniform case. All the essential difficulties are 
already present in the uniform model. 


1.2 Main results 

Theorems d] and [2] below state our main results. Figure [T] illustrates these when G is a path of 
length three, that is A: = 4. 

Theorem 1. Let G he a finite connected simple graph with \E\ > 2. 

(i) If 'X. G Xg, and n = nx -|- 0(1), then PG(n) ^ 0 exponentially fast, as n ^ oo, at a rate 
depending on x. The rate of decay is bounded away from 0 on subsets bounded away from TZg- 
(a) There exists a constant cg > 0, such that ifx G TZg, and n = nx-|-0(l), then pG{n) —)• cg, 
as n ^ oo. 
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(a) (b) 

Figure 1: (a) Image of pcirn, 200 —m—i, t) when G is a path of length three {k = 4) and n = 200. 
The limit of pc is a positive constant in the rectangle \ < x = m/n, y = Ijn < \ (dark region), 
and goes to 0 when (x,y) is away from the rectangle (white region). The maximum of pc is 
~ 0.2583299. (b) Detailed image of pQ near the corner of the critical region 0.15 < m/n < 0.35, 
0.4 < ^|n < 0.6. 

In Section [3] we obtain bounds on the behaviour near dlZc- These shows that the ‘critical 
window’ has width of order y/n around ndlZc- Our bounds in particular imply the following 
upper bound on Pg{t^) in this region. Fix any d > 0, and let 

Mn = Mn{6) = max{pG'(n) : n/re G So, dist(n/n, cIT^g) < • 

Theorem 2. For any 5 > 0 we have limsup^^go M„((5) < cq- 

Combining Theorems [1] and [2] we obtain the following corollary. 

Corollary 3. The configuration n that maximizes pcin) with n fixed, satisfies pcin) = cg+o(1), 
as n ^ oo. 

Theorems [1] and [2] do not rule out the possibility that PG(n) is maximized near the critical 
surface, at a distance that is o(n). But of course we expect that the location of the maximum, 
when rescaled by 1/n, converges to a point in the interior of TZg- It is also plausible that the 
location of this point can be characterized in terms of large deviation rates for events of the 
form ‘the gambler runs out of space on the edges in F\ that is: 

\ Y. 0CFCS. 

[w:deg^('u)=degQ(i;) i=l eSF J 
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We state an explicit conjecture for a path of length — 1, where this is easiest to formulate. Let 

k-j 


a*ij]k) = 




Ing- ( 


, l<j<k — 2 a*(0;A:) = 0 a*(/c — 1; fe) = 1. 


Let = (n™*™ : j = 1,..., A: — 1) denote a point in nSc where pg(ii) is maximized, n > 1. 
Conjecture. Let k > 3. Then for 1 < j < A; — 2 we have 

1 


hm - y^nj^ = a,{j;k). 

n^oo Ti ^ 


l=l 

The number a*(j; k.) is obtained as the unique point a G for which the ‘cheaper’ 

of the two large deviation events 

{ j n 'j I k n 

'^'^lvt=v > an\ and I ^ ^ lvt=^ > (1 - a) n 

tJ=l i=l J [^D=j;+2t=l 

is as ‘expensive’ as possible. (This number a can be obtained by equating the large deviation 
rates of the two events.) Each a.t{j;k) marks out a linear submanifold of 5 g, and the location 
of the optimum is their intersection. We expect that a similar characterization holds for any 
connected graph G. 

The structure of the paper is as follows. The proof of Theorem [1] is given in Section [2j We 
study the behaviour near &R.g in Section [3l and deduce Theorem [2l We stress however, that 
our analysis provides a much more refined picture than Theorem [2j see Propositions 11011111 and 
[13 and their proof. The estimates in these propositions suggest Gaussian behaviour near dlZc- 
We conclude with some further questions in Section [3 


2 Proof of the phase transition 

The next section collects some preliminaries and useful notation. 

2.1 Basic properties of TZg 

It will be convenient to have the version of TZg in which the inequalities are not strict: 

ICg = n X G Sg '■ for all F C E we have Xe > T^iF) i • 

I eSF J 

We denote hy Hp the hyperplanes appearing in these inequalities: 


Ff = J X G : V Xe = \d{F) I , 0 ^ F C F. 

I eSF J 


In particular, Sg, Fg, Fg and Kg are all subsets oi Hp- 
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Lemma 4. 

(i) The sets Kg and TZg are convex with a non-empty interior relative to He- 
(a) Kg = H-g (the closure of TZg in He)- 


Proof, (i) As intersections of halfspaces with He, both Kg and TZg are convex. Also, since the 
halfspaces defining TZg (resp. Kg) are open (resp. closed), TZg (resp. Kg) is a relatively open 
(resp. closed) subset of He- The containment TZg T Kg is immediate from the dehnitions. To 
show that TZg has non-empty interior, we check that the vector 


^* = {xl : e G E), x*. 


1 V—T 1 _ 

T y, '1 —rW) e G E, 
k ^ deg{v) 


(3) 


belongs to TZg- First, x* G He can be seen by summing the formula for x* over e G E and 
exchanging the two sums. It is also immediate that x* > 0, and therefore x* G Sg- Now hx any 
9 C F C E. Since G is connected, there exists a vertex v G V such that 0 < deg^(u) < deg( 5 (u). 
Therefore, 


e&F 


sis 

eSF v&V 


1 

deg(u) 


1 

k 


S 

vGV 

<iegp(v)=degQ(v) 


s 

e&F 

e'^v 


1 

degG(u) 


i y y_i_ 

k ^ ,ydegG(u) 

degp{v)<degQ{v) e~D 



S 

v&V 

degy(i;)=deg(3(i)) 


1 


d{F) 

k 


This shows that x* G TZg, and since TZg is open in He, X* is an interior point. The containment 
TZg C Kg implies that x* is also an interior point of Kg- 

(ii) Since Kg is closed, we have TZg C Kg- Therefore, it is enough to show that Kg \ TZg C 
TZg- Let x G Kg\TZg- Let x(t) = tx-|- (1 — t)x*. Convexity of Kg implies that x(t) G Kg for all 
0 < t < 1. Moreover, since the expressions 'Yhe&F ^e{t) are monotone linear functions of t, and 
^ggpXe(O) > d{F)/k, and X]eeF®e(^) — d{F)/k, we must have the inequality YIegf ^eit) > 
\d{F) for all 0 < t < 1. This implies that x(t) G TZg for 0 < t < 1, and hence x G TZg, as 
required. □ 


The optimality equation implies that the optimal deterministic strategy is also optimal 
among randomized strategies. The next lemma states a connection between elements of Kg 
and possible moves in a randomized strategy. In its statement, we think of q^^\e) as the prob¬ 
ability of assigning vertex v to the edge e in such a move. 

Lemma 5. IFe have x G Kg if and only if there exists a collection {q^'^\e) : v G V, e G E} of 
non-negative numbers such that: 

(V V G V; 

(ii) gH(e) =0 if e is not incident with v; 

(iii) i lOveV = Xe for all e G E. 

Proof. We deduce the statement from the Max-Flow-Min-Cut Theorem [21 Theorem III.I]. De¬ 
fine an auxilliary directed graph G' as follows. Replace each edge {u, w} of G by two directed 
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edges {v,Ue) and {w,Ue)-, introducing the new vertex for each e & E. Also add new vertices 
s and t. Add a directed edge {s,v) for each v G V and a directed edge {ue,t) for each e G E. 
Thus G' has \V\ + \E\ +2 vertices and 2\E\ + \V\ + \E\ edges. 

Consider flows of strength 1 from s to t in G', where we assign capacity 1/k to each edge 
(s,v), V G V, capacity 2 to each {v,Ue) and capacity to each {ue,t). 

Suppose satisfy (i)-(iii). Define a flow by letting 1/k flow on each (s,u), q^'"\e)/k 

flow on each {v,Ue), and Xe flow on each {ue,t). This flow satisfies the capacity constraints, and 
it is a maximal flow, since {(s,u) : v gV} is a cut with value 1. Therefore any other other cut 
must have value at least 1. Given ^ G E G E, consider the cut 

{(s,u) : deg^(u) < degciv)} U {{ue,t) :eGE}. (4) 


with value 


k - d{F) 
k 


+ Xe = 1 - 


d{F) 

k 


+ '^Xe>l. 


This implies that x G KLg- 

For the converse, suppose that x G JCg, and consider a maximal flow on G'. The conditions in 
the definition of ICg imply that all cuts of the form ([3]) have value > 1, and the cut corresponding 
to E = E has value 1. It is easy to check that any minimal cut is necessarily of this form, and 
therefore the maximal flow is 1. Letting q^^\e) be fc-times the amount flowing on {v,Ue) we 
obtain a collection satisfying (i)-(iii). □ 


Basic for Theorem [T] is the following computation. Suppose that our current state is n = nx, 
X G Sg- Let {g(’')(e)}^6v^,eeE be a set of probabilities representing a randomized move (that is: 

(v) 

qe is the probability that edge e will be used, conditional on the event that vertex v has been 
drawn). Let N' = (n — 1)X' be the random outcome of the move. Let ?/e = -^ Ylvev We 

have 


EX' = ^^EN' = ^ 


n — 1 


n — 1 



n 


n — 1 


-X — 


n — 1 


y = x + 


n — 1 


(x-y). (5) 


If X G TZg, then due to Lemma [5] it is possible to choose y G JCg in such a way that the average 
displacement points in any desired direction. On the other hand, if x G Xc, convexity of JCg 
implies that the process will always move away from TZg on average. 

The above observations are also reflected in the following deterministic controlled differential 
equation: 

— = X — u(t), where the control u satishes u(t) G JCg for all t > 0. 

It is easy to see (for example using as Lyapunov function the distance from HeCiHf for suitable 
E) that: 

(i) If x(0) 0 JCg, then for any control u we have x(t) 0 JCg for all t >0; 

(ii) If x(0) G TZg, then for any x' G TZg there exists a control u such that lim^^oo x(t) = x'. 









Let us introduce some further notation. Throughout we write ||w||i = Yle^E I'^el and |w| = 
V^eeE I'^eP for any vector w = [w^ : e € S) G For w G and A C we write 
dist(w, A) = infygyi |w — y|. We will write (•, •) for the Euclidean scalar product. 

For each 0 C F C E we fix a point G KLg such that Yle&E 
vector of the form 


ap if e G F; 

-bp ifeGF\F, 


with ap,bp > 0, and such that = 0. For all w G JCg we have (w — z^,u^) > 0. We 

will often use linear functions of the form: 


L^’"'(n) = (n — nz^, u^) = ^(rie — nz^)u^. 


e&E 


The last expression can be rewritten as follows: 


^(rie - nz^)u^ = ap'^Ue + (-bp) “ X/ ~ ni-bp) 1 - y~] 

e&F 


e&E 


eeF 


e&F 


- 

eSF 


{ap + bp) ^Ue- nbp - n{ap + bp) ^ z^ + nbp = {ap + bp)[Y^ 


rip — n 


d(F) 


eSF 


eSF 


\eeF 


We define k = k.{G) = min{(air + bp) : (Ji C F C E} > 0. We will need the following lemma. 

Lemma 6. There exist constants b = b{G) > 0 and B = B{G) such that for all w G JCg we 
have 

6 dist(w, cfFc) < min < Wp - i < F distfw, 5 Fg). (6 ) 

\ttp ^ I 

We also have 

< n [ Wp — ] < —F^’”(nw), n > 1. (7) 

Proof. The proof of Lemma [H^ii) showed that ICg \ B-g = QIZg- Therefore, if w G ICg \ TIg 
then X^esF^e ~ d{F)/k for some 9 C F C E, and dist(w,9FG) = 0. In particular, the first 
statement of the lemma holds when w G JCg \ TCg- Henceforth assume that w G TZg- Then 
since BIZg = U 0 c_pce-Hf H JCg, we have 


dist(w, 5 Fg)= min dist(w, /Cg H > min dist(w, n Fj?). (8) 

0C_FCE 0C7^C_E 

We claim that the last inequality is in fact an equality. Let F be a set for which the minimum in 
the right hand side of ([ 8 ]) is attained. Let wq be the orthogonal projection of w onto F£;nFi? in 
the linear space Fq. If the line segment wwq had any interior point wi belonging to any other 
HpG then this would contradict the minimality of F'. Therefore, the entire line segment wwq, 
apart from wq, belongs to TZg, with wq G &R.g- Hence dist(w, F^; n Hp) = dist(w, wq) > 
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dist(w, 97 ?.g). This proves our claim. Since w G He, there exists a constant Bq, that only 
depends on min{angle between He and He ■ 9 F <Z E}, such that 

dist{w, He) < dist(w, He n He) < Bq dist(w, He). 

This implies the first statement of the lemma, since dist{w, He) = (YleeP'^e ~ 

The second statement of the lemma follows from the definition of k{G), and the fact that 
aF,bE < 1 (since is a unit vector). □ 

Recall that we write n = nx for the starting state. Given a randomized strategy, we write 
Note that we allow the processes N(t), X(t), etc. to have negative entries, and 
once this happens, we have X(t) ^ Sq for all further times. We write Y(t — 1) for the vector 
of edge weights that our strategy prescribes for round t, and E{t) G E for the random edge 
selected in round t according to this strategy. We write 

Ht = a{N{s), Y{s) :0<s<t) 


for the filtration of the process. 

2.2 Steering 

In the following proposition we show that if n is large enough, then starting from any state in 
TZg that is bounded away from the boundary, there is a strategy that steers the process close to 
any other such point in TZg- 

Proposition 7. Given 5 > 0, there exist ci = ci{G,5) > 0, Ai = Ai(G,<5) > 0, no = no(G,<5), 
Ki = Ki{G, 6 ) and Gi = Gi{G,5) such that the following holds. Let n and ni be any positive 
integers such that n > (1 + Ki)ni and ni > uq. Suppose that n = nx with dist(x, 97 ^g) > b. 
Suppose also that z G TZg with dist(z, > <5, with niz having integer coordinates. There 

exists a randomized strategy starting from state n such that under this strategy we have: 

P[N(n — ni) = niz] > ci; (9) 

and for all q > 1 we have 

P [|N(n — ni) — niz| > q] < Ci exp(—Aig). (10) 

The strategy will be defined in three stages: in the first stage we reduce |N(t) — (n — t)z\ 
to 0(1); in the second stage we keep it within 0(1) until time n — ni — 0(1); and we use the 
last 0(1) steps to attempt to hit niz exactly. The first two of these steps are the content of the 
next two lemmas. After proving the lemmas we assemble them to prove Proposition [71 

Lemma 8. Given 5 > 0 there exists K 2 = K 2 {G, 6 ), do = do{S), X 2 = A 2 (G,<5) > 0 and 
O 2 = 02 (G) such that for any x, z with dist(x, 97 ^g), dist(z, 37 ^g) > b the following holds. For 
any n, n' with n > K 2 n' and n' large enough there is a randomized strategy starting from state 
n = nx such that the stopping time 

Tdo = inf{t > 0 : |N(t) - (n - t)z\ < do} 
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satisfies 


( 11 ) 


P^do > n-n']<C 2 exp(-A 2 n'). 

Proof. The value of do > 0 will be chosen in course of the proof. We are also going to use a small 
parameter 0 < eo < <5/4, chosen later. The first step of the proof is to reach an eo-neighbourhood 
of z. 

Let y be the point where the halfline starting at z and passing through x intersects OTZg- 
Let u denote the unit vector with the same direction as x — z. In the first step, we use the 
following strategy: given the current state N(t) = (re — t)X(t), we select Y(t) G dTZc such that 
Y(t) — X(t) is a positive multiple of u. In particular, Y(0) = y. We employ this strategy until 
the stopping time r(l) defined by 

r(l) = inf{t > 0 : |X(t) — z| < eo}- 

Let us write X°^*(t) for the component of the vector X(t) — z orthogonal to u. Let 

S{t) = (N(t) — (re — t)z, u). (12) 

Since 

N(t + 1) = (N(t) - Y{t)) + (Y{t) - l®(*+i)) , 
and the second term has mean 0 given J-), we have 

B[S{t + 1) I J)] = S{t) - {Y{t) - z, u). (13) 


Since x and z are bounded away from dTZc, there exist /x = /x(G, <5) > 1 and eo = £o{G, 6) > 0 
such that as long as |X°’'*(t)| < ^, we have 

(Y(t) — z,u) >/x|x — z|. (14) 


This implies that S'{t) = S{t) + tfi\x — z| is a supermartingale as long as |X°'’*(t)| < £o/2. On 
the other hand, due to the calculation in ([5|), X°''*(t) is a martingale. 

Let ti = Due to the choice of /x and eo, we have the inclusions 


{r(l) > ti} C {|X°'’*(s)| > eo/2 for some 0 < s < ti} 

U {S'(s) > fj.{n — s)|x — z| for some 0 < s < ti} U {S{ti) > 1} 

C I max |X°''*(s)| > 60 / 2 ! U | max S'{s) — S'{0) > (/x — l)re|x — z| 1 

(0<s<ti J (0<s<Xi J 

U / max S'{s) — S'{0) > - 

\^0<s<ti 

The inclusions (1151) imply 


-rex — z 


P[t( 1) >ti] <P 


max S'(s) — S'(0) > ——^relx — z| 

0<s<4i 2 


+ P 


max |X°’' (s)| > eo/2 

0<s<ti 


(15) 


(16) 
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Since S'{t) has increments bounded by (l + /i)-\/2, while |X“*(t + 1) — X“*(t)| < y/2/{n — t — l), 
we can apply the Azuma-Hoeffding inequality (see |14l Exercise E14.2] or [5l Theorem 12.2(3)]) 
to {S'{t)}t>o as well as to the projection of {X°'’*(t)}f>o to each coordinate direction. This 
yields 


P[r(l) > ti] < exp - 


(/X — 1)^ n^jx — zp 

8 2 (1 + ^)2 

< C' exp(—A'n) 


+ 2|E| exp - 


1 




(17) 


for some A' = A'(/x, Sq) > 0 and C = C'{G). 

For the second step we condition on the point ni = nixi = N(t( 1 )), such that n — ni <ti 
and |xi —z| < eg < <^/4- For ease of notation, we re-parametrize time for this step so that 
N(0) = ni. We choose Y(t) to be the point where the halfline starting at z and passing through 
X(t) intersects dTZc- Let us write u{t) for the unit vector with the same direction as X(t) — z. 
Decompose X(t + 1) — z = X'{t + l)u(t) + X"(t + 1), where (X"(f + l),u(f)) = 0. As long as 
|N(t) — (n — t)z| > do, we have 


|N(t +1) — {n — t — i)z| = + 

< (N{t + 1) - {n - t - l)z, u(f))2 + 2 


< (N(t + 1) - (n - t - l)z, u(t)) + 


do — 


Therefore, 


E(|N(t + 1) - (n - t - l)z| I Et) < E((N(t + 1) - {n - t - l)z,u(t)) | Jj) + 


do — \/2 

= (N(t) - (n - t)z,u(t)} - {Y{t) - z,u(t)) + 


do — \/2 


< |N(t) — (n — t)z| — J + 


do — 


Hence if we require that do > V2 + |, then 


D{t) = |N(t) - (n - t)z| + 2 ^ t > 0, 

is a supermartingale until Since the increments of D{t) are bounded by 2 + | < 3, and 
eg < |, it follows with t 2 = |ui that 


P [Tdo > ^ 2 ] < P 


max {D{s) - D{0)) > -ni 
0<s<t2 o 


< exp — 




64 • 32 12 


< exp{—X''ni) 


with some A" = A"(<5) > 0. 

Putting the two parts together, the statement follows if we choose K 2 = □ 

11 1 
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Lemma 9. Given J > 0 there exist A 3 = Xs^d) > 0 and C 3 = such that such that for all 

n' > n" > 0 and all w,z € JCg with dist(z, 97 ?.g) > d, |n'w — n'z| < do{d) the following holds. 
There exists a randomized strategy starting in state n' = n'w such that for all q > 1 we have 

P [|N(n' — n") — n"z| > (?] < 6*3 exp(—Asf?). (18) 

Proof. When |N(t) — (n' — t)z\ < do, let us apply an arbitrary move, otherwise, let us follow the 
strategy used in the second part of Lemma [SI We saw in the proof of Lemma [5] that 

D{t) = |N(t) - (n - t)z| + ^ X] - (n- 'S)z| > do] 

0<s<t 

is a supermartingale on any time interval s G [^ 1 ,^ 2 ) on which |N(s) — (n — s)z| > do- Assume 
the event 

F{q) = {|N(n' — n") — n''z\ > 4 ( 7 } , 

and suppose q > do- When n' — n” < q, the event F{q) is impossible, because |N(0) — n'z| < 
do < q and the increments of |N(f) — (n — t)z\ are bounded by 2. Hence we may assume that 
^max := [ip! — n'')/q\ > 1. Since D{0) < do < q, the inequalities 

\N{n'- n" - £q) - {n" + £q)z\ > Aq, £ = 0,..., £max, (19) 


cannot all simultaneously be satisfied. Summing over the smallest i for which (|19|) fails, we have 


p[m] < 

< 


E 


D{n' - 




^exp 


i>\ 


1 d'^q^e 
8 32 


n") — D{ri — n" — iq) > -qi 
^ < C'3exp(-A3g). 


( 20 ) 


Adjusting the constant Co, if necessary, we have the statement for all q > 0. This completes the 
proof. □ 


Remark. Note that the above strategy does not require the coordinates to stay positive. This 
will become important in Section [3.31 

Proof of Proposition^ Observe that if there is no point w such that dist(w, ST^g) > d, then 
the statement of the Proposition holds vacuously. Henceforth assume that <5 is small enough so 
that the set above is non-empty. We choose qo'>2 so that for the event F[q) introduced in the 
proof of Lemma Owe have P[T(go/4)] < Let M be the smallest integer such that 

M > (minirce : e G £1, w G TZg, dist(w, (97^g) > <^})~^ j 


which is finite by our assumption on d. We choose Ki and no such that n > ATini and ni > no 
imply n > 1^2 (^1 + Mqo), where K 2 is the constant from Lemma [HI Following the strategies in 
Lemmas [ 8 ] and [9] over the time interval [n,n — ni — Mqo] we have 

P [|N(n - ni - Mqo) - (ni Mqo)\ < g'o] > ^ - 6*2 exp(-A 2 ni) > ( 21 ) 
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if no is large enough. On the event in (j21l] we have 
Ne{n - ni - Mgo) - niZe 

> {Mqo)ze - \Nein - m - Mqo) - (ni + Mqo)ze\ 

> go - go = 0, ee E. 

Therefore, N(n — ni — Mqo) > niz componentwise, and there is a strictly positive probability 
Cl = ci{G,6) > 0 that niz can be hit exactly from the state N(n — ni — Mqo). This proves (l9|) 
of the Proposition. Since the form of the bound (IlSp is not affected by taking Mqo extra steps, 
statement pop follows from the estimates pil) and (I18p of Lemmas [5] and 0 □ 


2.3 Proof of the Main Theorem 

In this section we complete the proof of Theorem [TJ 

be a set such that 
have i ^e(O) < ^ Let 

= degG(u); 

0 otherwise. 


Proof of TheoremU^i). Fix x G Xq, and let 0 C F C Fi 
Then for some e = e(G, x) > 0 and sufficiently large n we 

Jl if 14 = u and degp(u) 


Since any v with degi;’(u) = degQ(v) must be assigned to one of the edges in E, we have 


PG(n) < P 


t=l e&F 


< P 




t=l 


< exp —n 


by Bernstein’s inequality; see O Theorem 2.2(1)]. The rate of decay is bounded away from 0 as 
long as X is bounded away from dTZc- □ 

Proof of Theorem[J)[ii) . We show that for any fixed 5 > 0 we have 


lim Mn = lim = a, 
n^oo n^oo 


( 22 ) 


where 


rrin = rnn{6) = min 


<PG(n) : ^5 
I eSE 


He = n, dist(n/n, SFg) > <5 > , n > 1; 


Mn = Mn{5) = max < PG(n) : Ue = n, dist(n/n, OTZg) > <5 > , n > 1; 

I e&E J 

a = a(6) = liminf mn((5). 


We consider n' > no, n > Kin' and n = nx such that = pG{n). We apply Proposition [7] 
with z = w'jn', where n' is chosen so that M^' = PGi^'). 
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Let (/?(r) denote the probability that with the strategy described in Proposition [7] the state at 
time n — n' is n'z + r, where Yhe&E = 0- Due to Proposition [TJ we have (^(0) > ci. Therefore, 
we can write 

mn=PG(n)> ^ tp{Y) pG{n'z + r) > cipcin'z) + ^ ip{r) pG{n'z + r) 

>':EeeB'’e=0 r^O: 

Eee-B 0 

> Ci{Mn' - rrin') + E (^(r) rrin' 

r:EeeB»'E=0 

> ci{Mni — mn') + rUn' — Cexp{—Xn') 


with some A > 0 and C depending on 6 and Ai, A 2 , A 3 . Rearranging gives 

1 C 

Mn' - mn' < —(mn - mn') H-exp(-An'). 

Cl Cl 

Since n> Kn' was arbitrary, taking liminf^^oo yields 

1 C 

Mn' — mn' < —(a — mn') H-exp(—An'). 

Cl Cl 

Taking limsup„/^oo in (l2H) yields Mn' — mn' —>• 0. Taking liminf„/^oo in yields 
0 < liminf(M„/ — mn') < —(a — limsupm„') < 0. 

n'^QO Cl n'^00 


(23) 


(24) 


This shows that lim„/^oo mn' = a, and the proof of (1221) is complete. 
The limit does not depend on <5, since for 0 < (5i < ^2 we have 


mn{ 5 l) < mn{S2) < Mn{62) < Mn{62), 
and hence Q!((5 i) = 0 ( 82 ) = cg- 

We conclude the proof by noting that cg > 0. This is because Proposition [7] implies that the 
process can be steered close to the point uqx* for a sufficiently large no with positive probability, 
and from here there is a strictly positive probability of winning. □ 

Remark. Since the left hand side of ()24p is non-negative, we can rearrange to get 

mn' < a + C exp(—An'), n' > no. 


We do not have a corresponding exponential lower bound on the speed at which the limit a is 
approached. See Question [T] in Section HI 


3 Upper bounds in the critical region 

In this section we obtain estimates in the critical region. This requires distinguishing a few cases 
that we state as separate propositions in the next section, and use them to prove Theorem [2l 
The proofs of the three propositions are given in Sections 13.2113.31 and 13.41 respectively. 
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3.1 Statements of upper bounds in three subregions 

We define the sets of configurations 

BQ{n]A) = {n G nSc ■ for some ^ C F C E we have L^’”'(n) < —A^/n"^ 

Bq {n;A) = {n G nSc ■ for all F with 0 < d{F) < k we have L^’”(n) > A^/n^ 

Bli\n-,A) = in G nSc '■ —Ay/n < min L^’"’(n) < Ay/n 
{ F-.0<dlF)<k 

Proposition 10. For all A > 0 we have 

limsup max{pG(n) : n G BQ{n-,A)} < exp 

n^oo 




In particular, the limsup is at most cq, if A > y/S^i^Jl/cG). 

Proposition 11. There exist constants C 4 ^ = C/^IG) and A 4 = A 4 (G) > 0 such that for all A> 1 
we have 

limsup max{pG'(n) : n G BQ{n]A)} < cq + Cieyrp{—XiA?). (26) 

n—^oo 

Proposition 12. There exists ^0 = ^o(G) such that for all A> Aq we have 

limsup max{pG'(n) : n G Bq^u] A)} < cq + C 4 exp(—A 4 ^^). 
n^oo 

Proof of TheoremlM assuming Propositions [HI EH Given e > 0, choose A sufficiently large 
so that each of the upper bounds in Propositions [101 [H [12] is at most cq + £• Since with 
this fixed choice of A the sets Bq, Bq and Bq^ cover all possibilities, the statement follows. □ 


3.2 Upper bound for Bq 

Proof of Proposition ESI We may fix the set F in the definition of BQ{n; A) and argue separately 
for each such set. Let us fix (5 > 0. Due to Theorem [T](i), we may restrict to n such that 

—6n< L^’"’(n) < —Ay/n. 

Let us follow the optimal strategy starting in configuration n. The process S{t) = LG”'“*(N(t)) 
is a supermartingale due to 

E[S{t + 1) I Ft] = S{t) - (Y(t) - z^, u^) < S{t). (27) 


Consider the stopping time 

T = ([n — cy/n\ + 1 ) A inf{t > 0 : S{t) < —5{n — t)}, 
where c = ^. Then we have 


< exp 


2 [n — Cy/n\ ) 
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Due to the optimality equation, pG'(N(t)) is a bounded martingale. Hence by optional stopping 
we have 

PG(n) = E[pG(N(r)); r < n - c^/n, S{t) < -5{n - r)] + E[pG(N(r)); t > n - c-v/n], (28) 

The first term in the right hand side of ()28p is at most 

max|pG(nO : ||n'||i > c-y/n, < —5n'| , 

which goes to 0, as n —oo, due to Theorem [T](i). The second term in the right hand side of 
(128)) is at most P[r > n — cy^ < exp(—< cq, due to our choice of A. This completes the 
proof of the Proposition. □ 

3.3 Upper bound for Bq 

We start with two propositions that strengthen Proposition [T) and will be used in the proof of 
Proposition [11] In the hrst, we give a lower bound on the probability that the process can be 
steered away from the boundary, if at least order y/n away. 

Proposition 13. There exist A 5 = X 5 {G) > 0, 7 = 7 (G) > 0, C 5 = 05 ( 0 ), C 5 = C^IG) and 
Uq = Ug(G) such that for all A > 1 the following holds. Let n,n' satisfy n'^ > n' > ng, and let 
n = nx be a configuration such that 

'^Xe> ^d{F) + for alUt} C F C E. ( 29 ) 

There exists a randomized strategy starting from n such that for the stopping time 

T = inf{t > 0 : dist(X(t), > C 5 } 

we have 

P[r > n — n'] < C 5 exp(—AsH^). 

Proof. Let y be the point where the halfline starting at x* and passing through x intersects 
OTZg. Write d = |x —y|, and note that d> due to Lemma [H Let r be the smallest integer 

such that {3/2yd > ^|x* — y|. We fix a small number rj > 0 such that ^ — rj > |. Then it is 
straightforward to check that the choice of r ensures that there exists 0 < 7 = 7 (G) < 1 such 
that — rjYn > if n > no for some no = no(G). 

Consider the sequence of points x = y(0),y(l),... ,y(r) dehned by 

y(0 =y+(3/2)*(x-y), f = 0, l,...,r. 

The following statement can be proved in essentially the same way as Lemma [H] For e > 0 
sufficiently small, there exists A = A(G, r],e) >0 such that given any point w G TZg with 
|w — y(i)| < e{3/2yd and any n such that — rj)n > no the following holds. There exists a 
randomized strategy starting in state nw such that for the stopping time 

r(i) = inf{t > 0 : |X(t) — y(f + 1)| < e(3/2)*'''^d} 
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we have 


P 


r(i) > 



n 


< exp (—A(3/2)^M^) . 


Summing the upper bounds on r(0), r(l),... ,r(r — 1) we obtain that there is a randomized 
strategy starting from state n such that for the stopping time 


t' = inf{t > 0 : |X(t) — y(r)| < e(3/2)’'(i} 


we have 

P[r' > n — n'*'] < Cexp(—A^^). 

Due to the choice of r, and for a sufficiently small £, the point X(t') is at least a fixed positive 
distance C 5 from dTZc, and hence t <t' . This completes the proof. □ 


The next proposition extends the result of Proposition [7| to the case when the target state 
is anywhere in fCc- 

Proposition 14. Given 5 > 0, there exists Ae = A6(G) > 0, Cg = Cq{G), cq 
Kq = Kq{G,6 ) and uq = nQ{G,6) such that for any ni > Kqu', n' > ng and 
ni = nix, X G TZg, dist(x, > <5 and n' = n'z, z G ICg the following holds, 

randomized strategy starting in state ni such that 

P [N(ni — n') = n'] > cg, (30) 

and 

P [|N(ni — n') — n'l > (?] < Cg exp(—Agg), g > 0. (31) 

Proof. We consider the following intermediate point; 

x" = -X + -x' and n" = n'x + n' + 0(1), 

where the 0(1) term guarantees that n" has integer coordinates. Observe that dist(x", (97 ^g) is 
at least a positive constant. Due to Proposition [7] we can steer the process from ni to a ((J/4)- 
neighbourhood of x" with probability at least 1 — Oi exp(—Ain'), provided Kq > 2Ki{G, 5). Let 
us call the point reached this way (2n')y". Since 

y" = x" + (y" - x") = l(x - 2(y" - x")) + lx', 

and | 2 (y" — x")| < |, the point w = x — 2 (y" — x") satishes dist(w, ST^-g) > |. 

Now consider the steps of the strategy of Lemma [9] for the starting state n'w and target 
state Ow over the time interval [0, n' — Mqq], where M > (minirce : e G i?})”^, and qq is chosen 
so that F{qo/A) > Let N(t), t > 0 denote this process. If the coordinates do stay positive 
until time n' — Mqq, there is a strictly positive probability of hitting state 0. When 0 is not hit 
exactly, we have the bound 

P[|N(n')| >q]= P[|N(n') - 0\ > q] < C 2 exp(-A 2 g). 

If we now apply exactly the same moves to the conhguration (2n')y", we obtain that the process 
N(t) = n' + N(t) hits n' = n'x' with positive probability, and satishes the bound in (I3ip . □ 


— ce{G) > 0 , 
configurations 
There exists a 
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Since the proof of Proposition [TT] is quite long, we first give a brief outline. Suppose we can 
select configurations n and n(£),..., n(l) in such a way that: 

(a) n/n is bounded away from &R,g-, so that we have pg{^) < cg + e; 

(b) n(f),..., n(l) are in the respective sets Bq with each pG(n(z)) close to the limsup in ([261) : 

(c) We can steer the process as follows: n —)■ n(.^) —)■ — 1) ^ • • • —n(l); 

(d) In each steering step we hit the target exactly with probability bounded away from 0. 

If i is large, step (d) ensures that PG(n) cannot be much smaller than the smallest of the 
PG{n{i)ys, and the claim will follow. The crux of the proof is parts (c)-(d), which rely on 
Propositions 1131 and 1141 The argument is somewhat delicate, since the n(z)’s now can be arbi¬ 
trarily close to SIZg', recall the definition of Bq in (j25l) . Therefore, Propositions 1131 and 1141 will 
be applied on a suitable subgraph that omits some edges. 

We carry out the plan (a)”(d). We start with some preliminaries. The first step is to 
subdivide Bq according to which part of OTZg is close. Given n E Bq , let 

g = g(n;G,A) = |Fci?:L^’"(n) < and F = ug, 

where n is the constant from Lemma El It may so happen that F = 0, in which case the 
arguments we have to make are similar to and simpler than when T 7^ 0. We will not spell out 
such arguments. Note that F £ Q implies d{F) = 0, since n E Bq . Hence we have 

(32) 

e£F F&G eSF FeS 


This implies d{F) = 0, for n large enough. Note that any F with d{F) = 0 that is not contained 
entirely inside F satisfies 

^ne> 

eSF 

Let us abreviate kq = k/ 2I®I'’'^. In the remainder of this section, we are going to fix a possible 
value To of T, and argue separately for each Fq. With this in mind we make the following 
definitions. For any Fq such that d{FQ) = 0, let 


By{n;A,Fo) 


|n E B^(i{n;A) 


Yhe&Fo < \Ay/n, and for all F not contained 
in Fq we have Yjc&f - ^d{F) > n^Ay/n 


Mn{Fo) = max {pG(n) : n e Bq (n; A, Tb)} 
fd = limsupM„(Fo). 

n^oo 


(33) 


Our task is to show that /3 < cg + C exp(—A^d^) for each Fq such that Bq (n; A, Fq) is non-empty. 

We will need to work on subgraphs of the form = {V, E^), where = E\H, H C Fq. 
We write for the restriction of n to , that is: = (ug : e E E^). When no confusion 

can arise, we will write = Yle&EH ^e- 

Lemma 15. If BQ{n;A,EQ) is non-empty, then for any H C Fq the graph G^ is connected. 


19 




Proof. It is enough to consider H = Fq. Should not be connected, we could write E = 
EiL) FqD E 2 as a disjoint union, where Ei and E 2 are non-empty and do not share any vertex. 
Then we have 0 < d{Ei U Fq), d{E 2 U Fq) < k and d{Ei U Fq) + d{E 2 U Fq) > k. Therefore, if 
n G By{n-,A,FQ), we have 

^ ^ rie + ^ He-'^Ue 

e£E eSEiUFo e£E 2 UFo e£Fo 

Tl 1 71 1 

> n -|- -A^/n > n, 

a contradiction. □ 

Lemma 16. Let H <Z Fq and n G Bq (n; A, Fq). 

(i) We have /n^ G KLqh . 

(a) Suppose in addition that Ue > cAy/n for all e ^ Fq \ H , with some c > 0. Then satisfies 
the assumption on the starting state of Provosition with A replaced by mm{cA, kqA} . 

Proof. Both statements will be proved by the same computations. Let 0 C T C (S \ F[). Since 
d{H) < d{FQ) = 0, we have d{F U F[-,G) = d{F] G^). When this common value is > 1, we have 

ne> Ue —-Ay/n >—d(F U H]G) + Ay/n —-Ay/n 

/ ^ ^ Zi k 2 

e&F eeFUH 

> "^diF-G^) + \Ayf^ > 

/C Zi K 

This already suffices for part (i). When d{F \J H]G) = d{F] G^) = 0 and F is not a subset of 
Fq, we have 

yy Ue > KoAy/n > kqAV. (35) 

eeF 

When ^ F F (Z Fq \ F[, under the assumption made in part (ii) we have 

yy Ue > cAy/n > cAVvW. (36) 

eeF 

The three cases (|3^ . (f35]l and (f36|) complete the proof of part (ii). □ 

The main technical difficulty in the proof of Proposition [11] is that we have no control over 
how small n^^i) can get for e G Tq) and therefore these coordinates must be hit exactly at each 
stage. We can do this, if the difference ne{i -|- 1) — ne{i) > 0 is sufficiently small so that we 
have enough opportunity to play these edges (once the exact value is achieved, we can ignore 
any such edge, since d{FQ) = 0. The configurations introduced next will help us overcome this 
technical difficulty. 
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Let x*’^o denote the configuration introduced in ([3]), with the graph G replaced by . 
Given 5 > 0 and H C Fq, let 


= (1 - + 6 - 


Fo\H\ 


E 1 '. 


eGFo\H 


where all vectors are regarded as being in Let = ny*’^°((5; Lf) + 0(1). 


Lemma 17. 

(i) We have x*Fo g }Cqh . 

(a) For all suffieiently small 6 > 0 we have y*Fo(^S] H) G TZqh and dist{y*FoH), dTlQH) > 
6 {B\Fo\H\)-K 

(in) There exists ct[G) > 0 such that for all suffieiently small (f > 0 and all $ C F C we 
have 

EniFm] 




e&F 


k 


Proof, (i) Let ^ F F <Z . We first consider the case when F (fi Eq\H and E\Fq (fi F . Then 
we have 


e&F eeF\Fo 


^ d(F\Fo;G^°) _ d(FU(Fo\F);G^) ^ djF'G^) 


(37) 


When F (fi Fq\H and F \ Fq C F, we have instead 


eSF eGF\FQ 


d{F-G 


H\ 


If 0 C F C Fq \ F, we have 


Y.x*/^ = d = 


d{F-G 




e&F 


(38) 


(39) 


This completes the proof of part (i). 

(ii) If 8 is sufficiently small, the inequalities (l37|) and ([381) . with x*’^o replaced by y*'^°{8] H), 
remain strict. Also, Eqn. (|39|) becomes a strict inequality. The lower bound on the distance 
follows from Lemma [6l 

(iii) This follows from (I37D . since the normalization factor in the front is [n(I — 0((5))]“^. □ 


Proof of Provosition 1771 Given e > 0, we select a subsequence along which M„(Fo) > (3 — e. 
For each n in the subsequence, select n G Bq (n, Fq) such that PG(n) > (3 — e. By passing to a 
further subsequence, we may assume that for each e G Fq the coordinates Ue are nondecreasing 
along the subsequence. 

We now choose n(l),.. . ,n{i) and n. Let n(l) < • • • < n{i) and let n(z) G Byin{i)-,Fo), 
i = 1 , he a sequence of points such that: 

(i) n{i + 1) > 2 ( 2 F 6 n(z))^/'>', i = I,... - 1] 
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(ii) ne(i + 1) > Tie(i), for all e G Fq, i = 1, V, 

(ill) PG(n(i)) > = 

We further define n in the following way. Let n = 2KQn{i), where Kq is the constant of 
Proposition [m and let n = Kq n{i) y *’^°+ Kqii{£) + 0(1) for a small > 0 for which 
the conclusions of Lemma fTW iil-fiiil hold. We will need that for all e G Fq we have 

rie < Kq n{t) + Kq\a \Jn{l) + 0 ( 1 ) < 25iKq n{t) = 5in, (40) 

l-^ol 2 

if n{tj is large enough. Also note that an application of Theorem [T](ii) yields pg{^) < CG + e. 

We now define the strategy to steer from n towards n(i). We first employ a strategy that 
plays an edge e £ Fq with Ne{t) > ne{i), whenever that is possible, but never plays an edge 
e £ Fq with Ne{t) = ne{t}. We stop the first time t when for all e £ Fq we have Ng,{t) = ne{Fj. 
Such a strategy exists, since (i(Tb) = 0. Since we start with iVe(O) — ne(l') < <5in (recall (IlQ|)), 
if (5i is sufficiently small, there is probability > 1 — exp(—An) that we stop before time CSn for 
some O = 0(0) and A > 0. Moreover, the value on every edge is decreased by an amount at 
most CSn, and therefore it follows from Lemma [1714 111 that the configuration n' reached has the 
property that (n')'^° is bounded away from . 

We can now apply Proposition [T3] to (n')^° and (n(l'))'^‘’ on the connected graph . We 
can implement the moves given by the strategy in that proposition as a strategy on G, because 
(i(Fo) = 0. Let ip£{r{i)) denote the probability that at time n{i) we reach state n(£) + r(^). Let 
us write ci = <^^(0) for the probability that n(£) was hit exactly. Note that since we applied 
the strategy on G^°, we have r’e(^) = 0 for all e £ Fq. This restriction will be implicit in our 
notation. Proposition [TT] implies 

CG + e > PG(n) > C£PG(n(^)) + ^ ¥?£(r(£)) pG(n(£) + r{i)) 

r{e)^o 

>Ci{P-s)+ ^ ipe{r{£))pG{n.{i)+r{£)). 

0<\r{£)\<uA^yn{i) 

with any n > 0. The value of n will be chosen in what follows. 

We now inductively define the strategy that steers from n(i + 1) + r(f + 1) towards n(i), for 
i = i — l,i — 2,... ,1. We assume |r(f + 1)] < vA^n{i + 1). Let 

H = {e£ Fq\ ne{i + 1) < S 2 Ay/ni+i}, 

where ^2 > 0 will be chosen in a moment. We will first reduce the edges in H to their target 
value ne{i). Then we use Proposition 11,41 and Proposition [7] in G^ to reach a target where the 
edges e £ Fq\ H do not have much excess compared to ne{i), so that these can be reduced to 
ne(i) as well. Following this, we use Proposition [TT] in G^° to hit n(z). 

The first part of the strategy is to reduce the value on each edge e £ H, whenever that 
is possible, until it equals ne(f), and in such a way that no edge m. Fq\ H is used. We stop 
the first time t when N(,{t) = ne(i) for all e £ H. Since d(Fb) = 0; such strategy exists. The 
goal is achieved before time CS 2 A^yn(i + l) with probability > 1 — exp(—Ay^ n{i + 1 )), if S 2 
is sufficiently small. Moreover, the value of every e £ E \ Fq \s decreased by no more than 
C52A^n{i + 1). Let n'(f + 1) denote the configuration reached. 
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Lemma 18. If 82 and v are sufficiently small, the restriction of the configuration n'{i + 1) 
to satisfies the assumption on the starting state of Proposition [13 with A replaced by 
min{^Ko^) 82A}. 

Proof. The proof is similar to the proof of Lemma [T6l Let ^ C F C E \ H. If d{F U H-,G) > 1, 
we have 

^<(i + l)= <(i +1) - ^ne(i) > Y <(* + 1) - + + 

eSF eeFUH eGH eGFUH e&H 


- 'Yl (^e(* + 1) + re{i + 1)) - {G + \H\) 82 Ay/n{i + 1) 

eeFUH 


> 

> 

> 


Y ne(i + 1) - + 1)1 - (C + |iL|) 52 ^\Ai(iTT) 


eeFUH 

UH-,G) + A^n{i + 1) - + (<^ + \H\) 82 )Ay'n{i + l) 

K 

G^) + (1 - G'u + G'' 82 )A^n'{i + l). 


(42) 


Hence we will require that 1 — G'v — G "82 > say. 

When d(F U G) = 0 and F is not a subset of Fq, we have 

Y. + 1 ) > + 1 ) + + 1 )) - G 82 A^Jn{i + 1 ) 

eSF eSF 

> Y ^e(* + 1 ) “ {VW\^ + G82)A-s/n{i + 1 ) 

T^f (43) 

> {kq - ■Ye'W - G82)A-s/n{i + 1 ) 

> ^KoAs/n'{i + 1 ), 

if n and 82 are small enough. 

Finally, if 0 C T C Fq\ H, we have 


Y, '^'ed + 1) = ^ ^e(* + 1) > ^ 82 A^/n{i + 1) > 82 A^/n'{i + 1). (44) 

e&F eeF eGF 

The cases (|42]l . ([431) and (1441) complete the proof. □ 

We need one more auxilliary configuration. Let n"{i) = 2KQn{i), where Kq is the constant 
from Proposition 1141 and let 

n"(l) = Ken{i)y*’^°{8i-,H) + {Kq - l)-^J^(n(i))^ + n(i) + 0(1). 

Due to Lemma [TTI iil. n"{i)/n''{i) E IZg and (n"(l))^/(n"(z))^ is at least distance c 8 i away 
from dlZQH . Therefore, we can apply Proposition [7] on the graph G^ to steer the process 
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from (n'(i + 1))^ to a ^3 neighbourhood of (n"(i))^, which succeeds with probability at least 
1 — Cl exp(—Ai(53n(i)). Moreover, due to Lemma [TTlfiiii. the configuration n"(i) + s reached this 
way satisfies 


{2Ken{i))-^ ^«(i) + + 4 , 0 C F C (45) 

e£F 

Also, for e £ Fo\ H we have 

+ Se) - ne{i) > Ken{i)yl’^°{Si-, H) - \/\W\\s\ - 

> Ken{i)-^ - 2KGn{i)s/\E\d3 - ^As/n{i) > 0 , 

I-cqI 2 

if 63 < (5i(4|Fo| y^|F|)“^ and n{i) is large enough. On the other hand: 

n”{i) + Se< Ken{i)5i + \/M|s| + KG^A-s/n{i){l + 0 {n{i)~^^^)) 

< KGn{i)6i + 2 KGn{i)^/\E \63 < 2Ken{i)6i, 


if n{i) is large enough. 

If is sufficiently small, we can now employ a strategy starting from state n"{i) + s, that 
reduces the values on all e £ Fq \ H, whenever that is possible, until they all equal ne(i), but 
never uses an edge in H. This only changes the values on e G F^° by at most 2C6iKGn{i), 
and succeeds with probability at least 1 — exp(—A2F'6n(i)). Let n'"(i) denote the configuration 
reached. It follows from (14^ that is bounded away from dTZQPQ. 

Finally, we can apply Proposition 1141 on the graph with starting state (n'"(z))^° and 
target state (n(i))^°. Let (pi{r{i)) denote the probability that at time n{i) we reach state 
n(i) + r(i). Let us write Cj = (pi{0) for the probability that n(i) is hit exactly. This gives the 
following inductive bound: 


PG(n(i + 1) + r(i + 1)) > CiPG(n(i)) + ^ ipi{r{i)) pG{n{i) + r{i)) 

r{i)f^O 

>Ci{(3-e)+ (^i(r(i))pG(n(i)+r(i)). 

0<|r(i)|<i/Ai/n7 


(46) 


Combining (|4ip and (I46p . Proposition I14I yields 

CG + e>{l3-£) [c£ + (1 - Ci)ci_i H-h (1 - q) • • • (1 - C2)ci] 

— C£exp(—AA^) — Cexp(—Ai^A-y/ni). 

Since each Cj > c > 0, we extract a factor arbitrarily close to /3 — e. Letting e | 0 shows that 
cg > /3(1 — e~'^) — C£exp(—AA^). Choosing i of order A^ completes the proof. □ 
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3.4 Upper bound for 


In the proof of Proposition [12] we are going to need the following lemma about supermartingales. 
It is a close variant of ( 8 ] Propositions 17.19 and 17.20] and hence we omit the proof. 

Lemma 19. Let Z{t) be a non-negative supermartingale with respect to Tt, and r a stopping 
time with respect to Ft- Suppose that 

(i) Z{t)) = k>l; 

(ii) \Z{t+l) - Z{t)\ <B; 

(Hi) there exist constants > 0 and b > 0 such that almost surely on the event {r > t}, either 
\lar{Z{t + 1) I Ft) > or Var(Z(t + 1) | Ft) = 0 and E[Z(t + 1) = Z{t) \ Ft] < —b. Then there 
exists ui = ui{B,b,a) and C = C{b,a) such that if u > ui then 

P[r >u]< C-^. 

\/u 

Proof of Proposition^!^ Given e > 0 choose Aq^e) large enough so that the conclusions of 
Propositions 1101 and 1111 are satisfied for all A> Aq. Under the optimal strategy, we consider the 
process 

Z{t) = min{L^'^-*(N(t)) : U, 0 < d{F) < k}, (47) 

which is a supermartingale, because the are. Since the increments of are bounded, 

condition (ii) of Lemmall9lis satisfied. We show that Z{t) satisfies the condition (iii) of Lemma 
[19] as well. Let F be the set contributing the minimum in (|47p . Since d{F) > 0, there exists an 
edge e G F such that gets updated with probability at least 1/k. On this event we have 

+ 1 )) - L^’”-*(N(t)) = -(!'=- z^, u^) =: - 6 (e; F) < 0, 

since d{F) < k. Therefore, if Var(Z(f + 1) | Ft) = 0, we have E[Z(t + 1) — Z{t) \ Ft] < —b{e;F). 
On the other hand, since there are only finitely many possible shifts in the values of the 
and only finitely many possible vectors Y(t) (recall that there exists a deterministic optimal 
strategy), if Var(Z(f + 1 ) | Ft) is non-zero, then it is bounded below by some = <j‘^{G) > 0. 
We will choose a small a > 0, and subdivide A) into the slices: 

Bq^u; a, /c) = {n G nSc : min {L'^’'^(n) : F, 0 < d{F) < A;| G [ak^/n, a{k -|- l)^/n)') , 
a > 0, femax 2 < A; < A:jiiax T Ij 

where /cmax = Let n G BQ^{n;a,k). The idea of the proof is to run the martingale 

PG(N(t)) until Z{t) moves well into one of the neighbouring slices, and use optional stopping 
to get an inequality relating the maximum of PG(n) over Bq^{ n;a,k) to the maxima over 
BQ^{n'; a,k — 1) and BQ^{n'; a,k-\- 1), with < n' < n. The parameter a will be chosen small 
so that we can apply Lemma [19] to the stopping rule. We will need to handle A:>1,A: = 0, —1 
and k < —2 separately. It will be convenient to introduce the following notation: 

Mn{k) = max{pG(n) : n G Bq^{ n;a,k)") 

Mn{k) = sup Mm{k) 

m>n 

/3{k) = limsupilU(fc) = lim Mn{k). 
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Case I < k < femax- We define the stopping time 


Tk = Van ( - - A inf > 0 : Z{t) < ~ aVn-t 

A inf > 0 : Z{t) > + 2 ^ a\/nW7| , 


It is straightforward to check that whenever Tk < n{^ — ^), the value of Z{Tk) is such that 
N(Tfc) is either in the slice EW'^ — 'T'k', a, k — 1) or in the slice Bq^ [n — Tk] a, A:+l). An application 
of Lemma [19] to Z{t) — {k — l)aVn yields 


< c- < C , ^ (48) 

By optional stopping, we have 
PG{n) = E[pG(N(rfc))] 

< P[Z{Tk) < kay/n - Tk\Mn/Vk - 1) + P[^('rfc) > {k + l)ay/n - TklM^/Vk + 1) (49) 
+ F[Z{Tk) G [kay/n - Tk, (k + l)ay/n - Tk)]Mn/Vk)- 

Note that due to our choice of a in (H5]l the probability in the third term of (]49|) is at most 
C{A)y/a. Maximizing pg{^) over its slice yields 


^ / 1 1 
Tk>Van(y-^ 


Mn{k) < Cn{k)Mn/Vk “ 1) + dn{k)Mn/VV + en{k)Mn/i{k + 1), l<k ^ ^max; (50) 

where dn{k) < C{A)y/a. By stopping the supermartingale Z'{t) = Z[t) — {k — l)ay/n at we 
have 

2ay/n > Z'(0) > E[Z'(rfc); Z'(rfc) > ^ay/n - r^j > ^ay/n\J 1 - y/aen{k). (51) 

When a is sufficiently small, the inequalties ([5T]) and dn{k) < C{A)y/a imply that Cn{k) > 

Case k = —1,0. We dehne 


Tk = -:an 
4 


We now have 


A inf > 0 ; Z(t) < “ 2 ^ ay/n — A inf > 0 : Z(t) > ay/n — . 

2ay/n 2y/aC 


Tk > -an 

a: _ 4 


< C 


ran 


V^v' 


(52) 


Analogously to ([501) we obtain 

Mn{k) < Cn{k)Mn/Vk - + dn{k)Mn/4{k) + en{k)Mn/Vk+ 1), A: = -1,0. (53) 

By an argument similar to the one for the previous case, for a sufficiently small we have Cn{k) > 
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Case —fcmax — 1 < /c < —2. This time we define 


Tk = ny/a 


1 


—A inf > 0 ; Z{t) < “ 2 ^ a^/n — 


l-k 4(1- 
A inf > 0 : Z(t) > + 2 ^ a\Ai| • 

Then with the same choice of a as in the case A: > 1 we have 


Tk> n 


This yields the relation 


1 - A: 4(1 - A:)2 


< C 


4a3/4 


2A 


(4-^) 


< C{A)^. 


Mn{k) < Cn{k)M^n{k - 1) + dn{k)M^j^{k) + en{k)M^/^{k + 1) ? ^max 1 ^ ^ ^ -2, (54) 
where Cn{k) > | for sufficiently small a. 

We select a subsequence of n along which Cn{k), dn{k) , en{k) all converge to some limits 
c{k), d{k) , e{k), as well as all Mn{k) converge to /3(A;). Then we get 


fi{k) < c{k)j3{k — 1) + d{k)/3{k) + e{k)f3{k + 1), 


(55) 


Due to Proposition [To] we have — 2) < e and /3(A:max + 1) < cg + £• It is easy to deduce 

from the relation ([55]) and c{k) > | > 0 that if j5{k) > j3{k + 1) then also j5{k — 1) > /3(A;). 
Hence the maximum in the variable k occurs at the right endpoint and /3(A;) < cq + £ for all 
—fcmax — 2 < k < kmax + 1- This completes the proof of the Proposition. □ 


4 Further Questions 

Question 1. It is plausible that the limit cq is reached at an exponential rate everywhere in 
TZg- If one could show that pci^^) is maximized in the interior of TZg: then this would follow 
rather easily from (|24|) . Can one describe the asymptotic behaviour of the optimal strategy? 

Question 2. The estimates in Section [3| strongly suggest Gaussian behaviour near OTZg- Can 
one make this more precise? 

Question 3. It is plausible that under the optimal strategy, the games starting from n, n' G uTZg 
(and with the same sequence of vertices drawn) couple with high probability. This may provide 
an alternative approach to the rather technical arguments of Theorem HKii) and Proposition fTTl 

Question 4. We describe a possible definition of an “order parameter”, in analogy with statis¬ 
tical physics models. Let 0 < a < 1, and suppose that the player has to give up proportion a of 
her/his moves to an adversary, at which times the move is chosen by the adversary. Let pG,a(n) 
denote the probability of winning in such a game. Let 

0(x) = inf{0 < a < 1 : lim pGa(^x) = 0}. 

n^oo ’ 

The methods of Theorem [Tj show that 0(x) > 0 in TZg and 0(x) = 0 in Ig- Can one analyze 0, 
or a suitable alternative? 
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